Evaluation of the TrueBeam machine performance check (MPC) beam constancy checks for flattened and flattening filter‐free (FFF) photon beams

Abstract Machine Performance Check (MPC) is an automated and integrated image‐based tool for verification of beam and geometric performance of the TrueBeam linac. The aims of the study were to evaluate the MPC beam performance tests against current daily quality assurance (QA) methods, to compare MPC performance against more accurate monthly QA tests and to test the sensitivity of MPC to changes in beam performance. The MPC beam constancy checks test the beam output, uniformity, and beam center against the user defined baseline. MPC was run daily over a period of 5 months (n = 115) in parallel with the Daily QA3 device. Additionally, IC Profiler, in‐house EPID tests, and ion chamber measurements were performed biweekly and results presented in a form directly comparable to MPC. The sensitivity of MPC was investigated using controlled adjustments of output, beam angle, and beam position steering. Over the period, MPC output agreed with ion chamber to within 0.6%. For an output adjustment of 1.2%, MPC was found to agree with ion chamber to within 0.17%. MPC beam center was found to agree with the in‐house EPID method within 0.1 mm. A focal spot position adjustment of 0.4 mm (at isocenter) was measured with MPC beam center to within 0.01 mm. An average systematic offset of 0.5% was measured in the MPC uniformity and agreement of MPC uniformity with symmetry measurements was found to be within 0.9% for all beams. MPC uniformity detected a change in beam symmetry of 1.5% to within 0.3% and 0.9% of IC Profiler for flattened and FFF beams, respectively.


| INTRODUCTION
Daily quality assurance (QA) testing of linear accelerators (linacs) is standard radiotherapy practice. In 2009, the AAPM Task Group 142 report 1 was published to supersede the AAPM Task Group 40 for recommendations on linac QA. The TG-142 report stipulates a daily linac QA program including testing of the photon beam output constancy.
The amorphous silicon electronic portal imaging device (EPID) has been used as a detector for linac QA measurements. [2][3][4][5][6][7][8][9][10] EPID is well suited to a number of linac QA tests as it provides a high spatial and temporal resolution two-dimensional digital measurement from a device that requires minimal setup time and is integrated into the linac. The latter allows for existing record and verify (R&V) databases to be used for data storage. Dose-response linearity [11][12][13][14] and reproducibility 15,16 of the EPID are also beneficial features for routine linac QA. A disadvantage of EPID as a detector for linac QA is the removal of the incident beam profile characteristics by the flood field correction. In addition, the high atomic number of the phosphor results in changed scattering properties compared to the equivalent depth in water 14,17 and also introduces an energy-dependent response. 13,18,19 Backscatter from the EPID positioning arm has also been a source of image nonuniformity. 20 Solutions to some of these issues have been developed including a backscatter absorber plate between the detection panel and the positioning arm with the aS1200 EPID. work, whereby the results of MPC were compared against other more routine monthly QA techniques. In this study, both MPC and the independent QA tests were run together on 10 consecutive days. From this dataset, the time required to perform MPC was investigated, and the mean and standard deviation was calculated for both MPC and independent QA measurements and compared.
The short duration of the study does not allow for any assessment of long-term stability and there is no measure of MPC sensitivity to machine errors, both of which are acknowledged by the authors.
Furthermore, the beam center constancy test is not compared against another QA method, and the QA test results are not presented in a way which allows direct comparison with MPC.
It is the aim of this study to compare the MPC beam constancy checks against both the departmental daily QA program and also to more rigorous monthly QA tests. This allows evaluation of MPC as a daily QA test device via direct comparison with current daily QA tests, and also provides an evaluation of MPC performance against more accurate monthly QA tests. The study was performed over a 5-month time period, which allows an assessment of the MPC stability and sensitivity to drift of the linac systems being tested. The 5-month measurement period was chosen to provide both greater than 100 MPC measurement points and also approximately 10 biweekly QA measurement points, which is the same number of measurement points used in the study by Clivio et al. 21 Sensitivity is further examined by the use of controlled modifications to beam output and symmetry and analysis of an annual QA event, whereby beam position was required to be adjusted for a single beam. The study provides routine monthly QA results in a form that is directly comparable to the equivalent MPC test. Along with output and uniformity, the beam center constancy is also evaluated.

2.A | Materials
All measurements in this study were performed on a single Varian

2.A.1 | MPC beam constancy checks
MPC is a closed system with minimal input from the user. The only parameters the user can adjust are the list of beams to be tested, the frequency of tests, and which measurement is to be used as baseline.
The individual tests, reported results, and thresholds are not able to be modified by the user. For the MPC beam constancy checks, the Iso-Cal phantom is retracted and the EPID set to 150 cm source-detector distance (SDD). Gantry and collimator are set to 0 degrees and an 18 9 18 cm 2 jaw defined field is used. Forty Monitor Units (MU) are delivered per beam energy using two distinct dose rates to incorporate a dose-rate constancy element into the testing. The raw (i.e., not flood field corrected) integrated images are analyzed. An up-to-date pixel correction map is required and from each image, comparison to the user-defined baseline image is used to determine changes in output, positioning of the beam center, and beam uniformity.
For the output measurement, the mean signal from a central region-of-interest (ROI) of 13.3 9 13.3 cm 2 is compared to the baseline. The use of this central ROI is to remove the effect of jaw positioning on the measurement. The output check threshold is set at AE 1.0%.
The same 13.3 9 13.3 cm 2 ROI is used for the uniformity constancy check. In this case, the ratio of the image ROI values to the baseline image ROI values is first calculated. A high-frequency filter is applied to remove high-frequency noise from the image and the result presented is the variation in the two pixels with the highest and lowest ratio from the baseline.
% Uniformity Change ¼ 100 Ã ðmaxðRatioðx; yÞÞ À minðRatioðx; yÞÞÞ (1) Only a single result is presented for uniformity, so it is unclear to the user whether a measured change in uniformity relates to the inplane or crossplane direction of the beam. The uniformity threshold is set at AE 2.0%.
For the beam center constancy check, the 18 9 18 cm 2 field edges are detected. From this, the position of the center of the beam can be determined and for each QA image, the position of this beam center is compared to the baseline position. Once again, a single result is presented and hence is not broken down into inplane and crossplane directions. The beam center constancy threshold is set to AE 0.5 mm. The linac parameters that influence each of the beam constancy check results are presented in (Table 1).

2.A.2 | Routine quality assurance methods
The Sun Nuclear Daily QA3 device (QA3) (Sun Nuclear Corporation, Melbourne, FL, USA) is used by the department for daily linac QA. In this study, QA3 dose, symmetry, and beam position tests are compared to MPC. A Farmer type ionization chamber is compared to MPC dose and the Sun Nuclear IC Profiler is used to measure beam symmetry and focal spot position. Finally, an in-house EPID-based QA program is used to measure beam position, dose output constancy, jaw position, and symmetry constancy.

Sun Nuclear Daily QA3
The Sun Nuclear Daily QA3 model 1093 running software version 2.4.1.2 is the current device used for daily QA beam constancy checks in our department. After alignment to cross hairs or lasers, data are acquired from a single 20 9 20 cm 2 field at 100 cm source to surface distance (SSD) and is compared to baseline. For this project, it is the dose constancy, symmetry, and beam position results that are of interest.

Farmer type ionization chamber
An IBA FC-65G 0.6 cc Farmer type chamber at 10 cm depth in solid water phantom at 100 cm SSD was used as the standard for output measurement. The chamber response was traced to the secondary standards laboratory, and response constancy checks using a Strontium source were performed quarterly to ensure consistent chamber response.

Sun nuclear IC Profiler
The Sun Nuclear IC Profiler is a 2D ion chamber array specifically designed for beam symmetry measurements. The IC Profiler can be attached to the collimator via a gantry mount and utilizes linear arrays of ion chambers. The IC Profiler has been previously characterized by Simon et al., 2010. 22 Besides flatness and symmetry, the IC Profiler also provides a beam center measurement. From the measured profile, the beam center is calculated as the midpoint between the 50% isodoses. When performed with 180 degree collimator rotation, the beam center measure can be used to determine the beams focal spot position.
In-house EPID-based quality assurance The same 10 9 10 cm 2 field at collimator 90 degrees is also used to assess dose constancy. A 9 9 9 pixel ROI is generated in the center of the field from which the mean integrated pixel value is recorded. This provides a measure that combines the output of the linac and any potential drift in the EPID response. The in-house EPID-based linac QA uses a 20 9 20 cm 2 field to assess profile constancy. From the same data, the accuracy of the symmetric jaw positions can also be ascertained. Profiles in both planes are extracted across the field. The coordinate system is normalized to the beam central axis from the center pixel measurement.
By comparing the position of the 50% isodose for each jaw to central axis, the accuracy of the jaw positioning is measured. The profile is then compared to the baseline profile (centered based upon the baseline center pixel measurement) to determine changes in flatness and symmetry. Absolute flatness and symmetry cannot be measured as the flood field correction removes the beam horns and any asymmetry present in the beam at the time. However, for the QA program, the flood field calibration is performed immediately prior to acquisition of the baseline images which are taken immediately after the beam is steered to best achievable symmetry. This means that each time the flood field is updated, the beam needs first to be steered and afterwards QA baselines reset. The experience of the department is that in the absence of an EPID fault, flood fields need only be updated annually.

2.B | Measurement methods
The MPC baselines were set for each beam energy. This was done following output calibration of the linac based upon monthly ion chamber readings and following beam steering based upon IC Profiler measurements. Measurements from all of the routine monthly QA methods were taken in the same session and also set as baselines. For every subsequent treatment day for the next 5 months, MPC was performed alongside QA3. Additionally, the routine monthly QA tests as outlined in Table 2 were performed on a biweekly basis. MPC results were compared both to QA3 and routine monthly QA tests over the 5-month period. MPC short-term repeatability was assessed by performing five successive measurements on two different days and calculating the standard deviation. The detector was not moved each day between measurements.
For the beam center constancy and uniformity constancy comparison, the routine QA results were presented in a form most directly comparable to MPC. For the beam center check, the routine QA inplane and crossplane results were both determined. The plane with the greater deviation was compared to MPC.
Using the IC Profiler, the focal spot position was calculated using its inbuilt beam center parameter and measurements from collimator  Once the IC Profiler symmetry had been verified, the four beams

3.A | MPC beam constancy repeatability
The repeatability measurements showed that output was repeatable to within AE 0.03%, the uniformity to within AE 0.1%, and the beam center to within AE 0.04 mm. Before repeatability measurements were performed, the baselines were reset. It was assumed that the results immediately post baseline reset would come back close to zero. In the 16 measurements (four measurements for each of four beams) taken for repeatability, it was found that the beam center The results of Fig. 1 show a steady increase in output across all measurement methods. Figure 2 presents the same data as Fig. 1, but in the form of measured differences between MPC and each of the other measurement methods.

3.C | MPC output sensitivity
The results of Fig. 1 show how the MPC output is sensitive to gradual changes over time. The result of Table 3 shows how sensitive the MPC output is to a large sudden change in output. Table 3 shows that for a Monitor Chamber output adjustment of approximately 1.2%, the MPC measured output change is in agreement with ion chamber to within absolute difference of 0.17% for all energies.

3.D | MPC beam center
The results of Fig   and whether linac parameters are working in opposite directions and hence canceling each other out. Figure 5 indicates that in the majority of the measurements, it was the jaw positioning that was the greatest contributor to beam center variation.

3.E | MPC beam center sensitivity to change in focal spot position
The results of Table 4

3.F | MPC uniformity
The results of Fig. 6  The results of Fig. 7 show how well the MPC uniformity measure compares to symmetry measurements from commonly used QA devices over a period of 5 months. The QA3, IC Profiler, and inhouse EPID symmetry measurements have been presented as inplane and crossplane added so that a meaningful comparison with MPC uniformity is made. Figure 7 shows no overall trend, so a comparison of the mean and standard deviation of the data is presented in Table 5 for all energies. Figure 7 and Table 5  Note: 1 SD refers to one standard deviation in measured results over the 5-month period of measurements giving an indication of the day-to-day variability.
F I G . 6. MPC uniformity results for the four beams (6 MV, 10 MV, 6 MV FFF, and 10 MV FFF) over the 5-month measurements period.  and ion chamber results differ by more than 1%. The ion chamber response is checked with a strontium source every quarterly and no drift was detected in these checks performed before, during, and after the measurement period suggesting that the ion chamber results can be relied upon during these measurements. This suggests that it is MPC drifting in response rather than the other methods.
The divergence is also apparent when comparing MPC to the inhouse EPID measure. As both of these methods utilize the EPID as the detector, the differences in the EPID acquisition would appear to be the source of the divergence. These differences include EPID calibrations including that MPC is not flood field corrected and potential difference in the dark field application, field size and ROI size differences, mode used for acquisition (in-house EPID using treatment mode, MPC using its own MPC major mode), the varied dose rate used for MPC rather than the constant dose rate for the in-house EPID. There is no clear evidence, which if any of these acquisition differences might be causing the divergence.

4.B | MPC beam center
The results of Fig. 3 do not indicate a trend in the MPC beam center

4.C | MPC uniformity
The uniformity results of Fig. 6 demonstrate a systematic offset from the baseline present in all four beams. As the uniformity is measured by taking the ratio of the measured image against the baseline image, it would be expected that images taken immediately after the baseline would provide a result close to 0%, which may drift away from 0% over time. This is not evident in Fig. 6. Measurements taken immediately after resetting the baseline indicated an average 0.5% systematic offset in uniformity. This is unexpected and not within measurement repeatability (AE 0.12% 1 SD). No explanation is provided for this offset.
The statistical disagreement between MPC uniformity and QA3, IC Profiler, and in-house EPID measured combined symmetry as presented in Table 5 will be influenced by the systematic offset measured with MPC uniformity. If the measured offsets are subtracted from the results of  Table 6 show a systematic insensitivity of MPC compared to IC Profiler. The results for 10 MV FFF (Table 6) show even less agreement than the 6 MV FFF beam, but as explained previously, the 10 MV FFF beam had greater magnitude of steering then the other beams and this distorts the results.
The effect of beam angle steering on the FFF profiles was found to be different than the flattened beams. For flattened beams, the expected effect of a tilting of the profile was observed, while for the FFF beams, the most noticeable change was a lateral shifting of the peak. In both cases, shifting of the positioning of the penumbra was negligible. An example of this is shown in Fig. 8. This behavior may provide an explanation as to why the MPC uniformity sensitivity does not agree with combined symmetry measurements as well for FFF beams as it does for flattened beams as shown in Table 6. The combined symmetry principle works on a simplistic assumption that when beam angle steering is applied, one side of the profile decreases by a certain magnitude and the other side increases by the same magnitude. When the two magnitudes are added, the symmetry value is obtained. Using this principle, the measured asymmetry in both inplane and crossplane can be added and this combined measurement will show the expected MPC nonuniformity of the beam if the nonuniformity is caused by beam steering (asymmetry) alone (i.e., beam energy and EPID relative detector response constant). However, Fig. 8 demonstrates that the effect of beam angle steering on FFF beams is not to reduce one side of the profile by an amount which the other side increases by, but to shift the peak laterally. This behavior can explain why the results of Table 6 do not show agreement between combined symmetry and MPC uniformity for FFF beams. This demonstrates that the combined symmetry method is not suitable for comparison between symmetry measurements and uniformity and that a different method for comparison needs to be found.
The shifting of the peak for the FFF beams contributed to initial difficulties mis-steering the 10 MV FFF beam. While the other three beams were mis-steered by 1.5%, the 10 MV FFF beam was mis-steered by up to 4.2%. This was because the IC Profiler was originally programmed to set central axis for IEC symmetry determination by using the average position of the 50% points on the profile. Due to the peaked nature of the 10 MV FFF dose profile for a large field then for the 30 9 30 cm field the points at 50% of central axis lie within the umbral region of the profile rather than on the penumbra. As the beam was being mis-steered and the dominant affect was to shift the position of the peak this also shifted the 50% points and hence made the symmetry readout insensitive to the changes in beam angle steering so that when the symmetry measurement recorded 1.5% the actual symmetry using center of collimator rotation as reference was up to 4.2%. This was not a problem for the 6 MV FFF beam as for this beam at 30 9 30 cm field size, the 50% isodose lies on the penumbra. The obvious lesson is that with larger field sizes for FFF beams, the average position of the 50% isodoses is not a good indicator for determination of center of the beam.
The instance where MPC recorded a 20% output deviation for the 6 MV beam immediately after the EPID panel had power restored is a reminder that when used for dosimetry and QA, the EPID panel is a detector susceptible to fault like any other. If MPC is to be relied upon for daily linac QA, then a QA program for the imaging systems and MPC specifically is required appropriate to their use as QA detectors.
Over the 5-month measurement period (n = 115), drift in the results was not detected in the MPC beam center or uniformity results, which allowed meaningful mean and standard deviation values to be calculated. Clivio et al. 21

| CONCLUSION
The three MPC beam constancy tests have been evaluated against daily QA and monthly QA procedures over a period of 5 months.
Each MPC test has also been tested for sensitivity to appropriate changes in the linac beams that they could be expected to detect.
The beam output and beam center tests have been found to be at least equivalent to routine daily QA procedures and in some ways superior. A drift in MPC output was observed that suggests that regular intercomparison of MPC output with an ion chamber is required.
This is now performed monthly in the department. The uniformity test was found to give a result offset from zero in measurements taken immediately after resetting the baseline. Uniformity was found to be accurate and sensitive to changes in beam symmetry for the flattened beams, but not so for the FFF beams. The different behavior of the FFF dose profile compared to the flattened beam profile with changes in angle steering is thought to be the cause. For the beam center and uniformity tests, the results of Clivio et al. 21 have been compared with a larger dataset with general agreement observed. In our department, we have now replaced the Daily QA3 with MPC beam constancy checks for daily linac QA. It is recognized that the insensitivity of MPC to changes in FFF beam symmetry is a limitation, but this check is not required by AAPM TG-142. It is recommended that improvements could be made to the MPC beam constancy checks by breaking the results for beam center and uniformity into inplane and crossplane, presenting uniformity in terms of flatness and symmetry constancy and modifying the beam center measurement to take out the effect of jaw positioning, which is tested in the MPC geometric tests. It was also discovered that the effect of beam angle steering on the profile shape of the FFF beams differs from the flattened beams and consideration must be given to the metric used for beam center determination when beam steering large field size FFF beams.